{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BP神经网络\n",
    "### BP神经网络也叫做误差反向传递网络\n",
    "> 引用 https://blog.csdn.net/daaikuaichuan/article/details/81135802\n",
    "### BP神经网络基本原理图：\n",
    "![image-2.png](https://img-blog.csdn.net/20180720162033746?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhYWlrdWFpY2h1YW4=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### BP神经网络分为输入层，隐含层，以及输出层。如上图：\n",
    "\n",
    "- 对于输出层节点i：\n",
    "  $δ_i = y_i(1-y_i)(t_i-y_i)$\n",
    "- 对于隐藏层节点$δ_i$:\n",
    "  $δ_i = a_i(1-a_i) \\sum_{k=1}^n W_{ki}*δ_k$\n",
    "最后，更新每个连接上的权值：\n",
    "- $W_{ji} \\leftarrow W_{ji} + η * δ_j * X_{ji}$\n",
    "其中η为学习率(学习率过大和过小都不适合拟合)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### BP神经网络推导基本过程：\n",
    "1. 我们取网络所有输出层节点的误差平方和作为目标函数：\n",
    "- $Error = 1/2 * \\sum (t_i - y_i)^2$\n",
    "\n",
    "误差平方和越小，说明拟合度越高，越接近于真实情况。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. 求出误差后，我们需要将误差反向传递，修改每个节点间的权重，我们运用随机梯度下降算法对目标函数进行优化：\n",
    "\n",
    "- $W_{ji} \\leftarrow W_{ji}−η * \\partial E_d / \\partial W_{ji}$\n",
    "\n",
    "3. 根据链式求导法则，可以得到：\n",
    "-  $\\partial E_d / \\partial W_{ji} = \\partial E_d / \\partial net_j * \\partial net_j / \\partial W_{ji}$\n",
    "-  $\\partial E_d / \\partial W_{ji} = \\partial E_d / \\partial net_j * \\sum_{j=1}^n \\partial W_{ji} * X_{ji} / \\partial W_{ji}$\n",
    "-  $\\partial E_d / \\partial W_{ji} = \\partial E_d / \\partial net_j * X_{ji}$"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 这里权重的推导需要区分从输出层到隐含层的权重和隐含层到输入层的权重，对应链式求导有不同。\n",
    "4. 对于输出层到隐含层：\n",
    "- $\\partial E_d / \\partial net_j = \\partial E_d / \\partial y_j * \\partial y_j / \\partial net_{ji}$\n",
    "其中$y_j$为$net_j$的函数，即$y_j=sigmoid(net_j)$，并且有$y_j$的一阶导数为：\n",
    "- $y_j^{-1} = y_j·(1-y_j)$"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. 对于隐含层到输入层：\n",
    "- $\\partial E_d / \\partial net_j = \\sum_{k=1}^n \\partial E_d / \\partial net_k * \\partial net_k / \\partial net_j$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是神经网络的大概流程：输入→隐含层→输出→计算误差(均方差)→误差函数反向传递→计算各层之间的权重更新值（运用梯度下降）→反向传递，如此重复训练，设定误差范围，最终能够找到最优的拟合或分割曲线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BP神经网络实现Iris数据集分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error= 0.2121268391601158\n",
      "测试分数为: 0.9666666666666667\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEICAYAAABVv+9nAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABMn0lEQVR4nO3dd3hUx9X48e9sVUcIBEiid0SvpoPBgLGN44or7uZ1bOfnkrxx6hvixOmJEycusXEnLrjGDWNjejO9F1EFSGCBEOrSlju/P+5KbFNbrbRtPs+jB2nLvUdCOjt77swZIaVEURRFiT6GUAegKIqitAyV4BVFUaKUSvCKoihRSiV4RVGUKKUSvKIoSpRSCV5RwoQQwhjqGJToohK8ogRACHGdECLN9fl0IcSTjXjOJUKIJ9y+/kgI8X+uzxOArUKIWS0WtBJzVIJXlCYSQvQA3gUmu27KA54QQgxo4KllwGNCiF+7vq4GKlyfP+f6ekOQw1VimErwSkQRQtwlhJCuD6cQ4pgQ4lHXfSvd7isXQqwRQozyet50t2O9JoRY6fbcY+5lEtfj7/ITxs+Ab6WUHwNIKQ8Ai4A3hBAmPzGbhBDxwH5gDjBVCGF1u78XMNx1X7kQIlEIIZrzc1IUUAleiUzngbZAe+D7wO+EEJe77vuX675BwBbgcyFEG7fn/rie43YH5tZ3YiHEJOBu4Idedz0BdAFe8JOcbwDKgRLga2Ao+mj+JuA3wFagG5ADFLm+v7T64lCUxlAJXolEUkp5QUpZJKX8EtgLjHPdV+267xjwIyDV7T4nMFMIMbSO4zqB/63rpK4XileBF6SU33oFdA64DrgZeM/rRWWxlNIgpUySUqYCdwIFwHbgl8DfgM+BwVLKFCmlVUpZ2KifhKLUQyV4JaIJIYYD2cAOP3dr6Enb7Pr6ArCMukfxHwGDhRAz/JwnDvgAcAAvCiF6CCG6u38A+cA9wCRgjxDiCgAppeY6RnchxKvAX4ArgOOu2J5GT/h7hBA/FUKYUZQgUAleiURpQogLQogSYBvwipTyI/cHuGrhj6In5I1ud/0BmCuE6ObnuLnAO/h/Afgl0B94EtgJHAWO+fl4BRgJHASOuGJJEUIsATa5njdESrkTvUxTLKUslVI+BswE7geeatJPQ1Hq4HNBSFEiwAVgBProdzzwTyFETRJ/WAhxH5CEPrtlrpTybE1ZXEr5jRBiO/B4Hcf+I7DL9c7A3f+hz3Q5DbyHXq/PATpIKc8CCCHuAJ6QUp4CLnN7bgX6qP1bKWWZ2+3lwDTXcZFSfiuEGAZoQgijlNLZ6J+IovihErwSiTQp5XHX54eEEGPQL3qWAG8AvwPKXHVxf/4IvA4s975DSrlHCPE5XqN4V7LNqzm/a3bOiZrk7tIFOOHnfL8CfgJUCyEcbrenuI51we02ExAH3OuKUVECpko0SjQQXKyzl0gpj9eT3EGvteeh18H9+SNwYwPnvBv4r9dtfhO8lPKXUkpzzUVW14XW24HvXHE8WHO76zEmKaVK7kqzqQSvRCIhhEgVQnQUQnwPPVl+09gnuy56/hnw2xpASrkWz7q998kfB8ail13cdcX/CN77+ROBF4H/h/4i82chxI9VqwIl2FSCVyJRGvp88TzgGeDf6IuPmuIN9FkvdfmD9w1CiPZCiOfRL7ReL6U84bp9qGse/gRgl7+DCSGsQohLhRCL0KdE/lBK+Z6Uci/6dYQbgH1CiMdcK2UVpdmE2rJPURomhOiEviDpCPB9V2Kuue/PwPXA28CvpJQOr+feiL7StdD171+klAVejzGgz4//MfoF3EFSyiMt9g0pMUEleEVpJCFEZ9cMmaY+z4TeimCLbMQfnBCim5QyN5AYFcWdSvCKoihRStXgFUVRopRK8IqiKFEqbBY6CdFe6teWwoNAYzjb8dezVUOwnRGtHpPScvr23RrqEJQgKy3NCHUIreL06dPnpJTp/u4Lmxq8EKOk3t01XEiO0YPu+F7r2sBYxqt9GaLKihWq/Xq0WbVqQahDaBULFizYKqUc5e8+VaKpk+AR/k45CbW3aEA5CfyQv4YuLEVRlEZSCb4en3ANV/EZK5lMPhl8yeVMYRUbGB/q0BRFURoUNjX4cLWSS1nJpaEOQ2lBqjwTXWKlNNMYagSvKIoSpdQIXlGUqKBG7r7UCF5RFCVKqRG8ErNU7V2JdmoEryhKxFPlGf9UglcURYlSqkSjKIoSIiUlSWzaNIa8vCw6djzDJZdsom3b4qAdXyV4RVEiViSXZs6ebcfChffhcJhwOk3k5nZl27aR3HXX62Rmng7KOVSJRok5K1YIdYFVCbkvv7yc6moLTqc+ztY0Ezablc8+uzJo51AJXlEUJQSOH++OvxScn5+J0xmcAYgq0SiKEnEiuTRTw2KxUVnpm4JNJgcGQ3C6/KoRvKIoSgiMGrUFk8nucZvJZGfo0J2IIFUQVYJXFEUJgSlTVtG3bw4mkx2rtQqTyU63brnMmrU0aOdQJRpFUSJGNJRmaphMGnPnvk9RUSpnz6bTrl0h7dqdD+45gno0RQlzavaMEm7atr1A27YXWuTYqkSjKEpEiKbRe2tRCV5RFCVKqQSvKIoSpVQNXlGUsKZKM4FTCV6JCeriqhKLVIlGURQlSgWc4IUQHYUQa+q53yyE+EwIsV4IcU+g51EURVECE1CJRgjRFngdSKznYT8AtkgpFwghPhRCvCelLA3kfIqixB5Ve2++QEfwTuAmoKSex0wFFrs+Xw+MCvBciqIoSgACGsFLKUsARP0dcRKBPNfnJUBH7wcIIeYD8/WvugYSiqIoUUiN3oOjJWfRlAHxQDGQ5Prag5TyReBFACFGBac/pqK4UbNnlFjWkrNotgITXZ8PBY634LkURVEUL0EZwQshpgHZUsp/ud38OvCFEGISkA18G4xzKYoSvVRpJriaNYKXUk51/bvcK7kjpcwFZgDrgMuklM7mnEtRFEVpmhZdySqlzOfiTBpFURS/1Mi9ZaiVrEEhGcO3XMWntOdsqINRXNQFViXWqV40zdSVXL5mBhmcxokRK9X8iR+zgF+HOjRFUWKcGsE306fMoSdHSKaMVIqJp4of8leu4tNQh6YoEUGVZ1qOSvDN0I8D9OIIJjSP25Mo5wc8E6KoFEVRdCrBN0MKJTjqqHKlUdTK0SiKoniKuRp8OgVM5xvKSeQrZlJNXMDH2slQBL4LcCuI4wOua06YiqIozRZTCf4Rnub3/Aw7ZiQCDcGVfMEGxgd0PBtWHuB5XmI+VqowoVFOAqfozLM8HOTolcZSs2cig6q9t7yYSfAj2cJT/IJ4qoinqvb2z7mCTnyHDWtAx32b29jHQB7mX2SRx+dcyavcTUW9nZQVRVFaXswk+Ht4GatbYq9hQGMmX/EZcwI+9k6GcT8LmxOeosSMSBi5O50Cu92M1Wqj/qa54S1mEnwyZT6zXQAEkEh56wekKErYcTgMfP31DLZuHYmmGUhOLuWKK5bQr19OqEMLSMzMovmA6yn1UzYxY2cZl4UgIiXYVqwQqv6uNMvnn1/J1q0jcDjMaJqR4uJU3n//ek6c6Bzq0AISMwn+U+awkqmUkgSAAwPlJPAEf6CQ9iGOTlFiQziXZyorrezaNQSHw+Jxu91uZvXqySGKqnlipkSjYeR7fMJVfMYNvE8JKbzCPWxnRKhDUxQlDJSWpmA0OnA6vdOioLCwXUhiaq6YSfAAEgOfcjWfcnWoQ1GUmBLOI/caqakXkNK3qCGERmZmfggiar6YKdEoiqLUx2KxM378Osxmm9utEpPJwZQpq0IWV3PE1AheURSlPlOnriY5uZR16yZSXp5IVlYeM2d+TYcO50IdWkBUgleigpo941+FEYoskF4NFt9ZwooXIWDUqO2MGrW9zsdoGpw/3w6z2U6bNiWtGF3TqQSvKFHIIeCfveHLTmB0tUualws3n9TXfrSWSKi9N8WRIz346KNrqa62IqUgPf0sN920mNTU4lCH5peqwStKFPp3T1jaCWxGqDTpH290h686hjqyyFVUlMo779xMWVkydrsFh8PMmTMdee21O9HC9N2RSvBu+nKQm3mbS9gIfrpEKkokcAj4LBOqjZ63VxlhUbfQxBQNtmwZidPpmTKlNFJRkUBubvfQBNUAVaIBTNh5h5uZzRIcmBBoHKUX0/lGLYJSIk6VUU/y/py3+L+9JURbeaa4uA2a5j9llpYmt3I0jaNG8MAP+QuXs4QEKkmhlGTKGcB+XuOuUIemNEC1J/CV6IA2dj93SOgf3tcEw1rPnse8plDqNM1A586nQhBRw1SCB77PCyRS6XGbBTsz+YpEykIUlaIERgA/OARWp9ttEuI0+J+jLX/+VasWRN3oHWDw4N2kpJRgNDpqbzObbQwatIe0tPDcwS0GSzSSKaziBt7DjoVF3E48FXU8UmClmnJX/xpFiRRTzkHKbni9G+THQ99SuPs49FKNUwNmNju4//6FrFs3nn37srFYbIwevZlhw3aEOrQ6xViCl7zIfG7hbRKoQCKYz4vspz9tuYAZh8ejj9CL80RmDwpFGX5B/1CCJy6umunTVzB9+opQh9IoMZXgx7OeW3iLpNoRuySRCoaxAwFoCAxIqrBgx8LdvBrKcBUlokRjWSbSxVSC/x4fk+BVawcwotUu/nBg4Bg9mMEy8ojMHtCKoigQYwm+kgScGDF4lWLc52CY0OjGCZIpbd3glCZRM2cUpWExNYvmLW7FjrnBxzkxMootrRCRoiix5MyZjnzxxeV8+OE17N/fH01r2YFKTI3gc+jH4/yNp3kMByYs2LBg8+nNIYHjdA9BhIoSmVT9vWGbN49k6dJZOJ1GpDSwf/8AunY9wW23vYXB0DIr52NqBA/wbx6gG7k8xLM8xt8o89qn1YaJU3RhLRNDFKGiKNGmsjKOpUtn4XCYazcVsdstnDjRhf37+7fYeWNqBF/jLB14kzsA2Moo3uIWunICgWQHw7ma/9K6PfcUJXScApBgbPCRvtTIvXGOHeuO0ejE4fAsEdvtVvbuHcjAgftb5LwxN4L3NoD9ZHAG6Zoimc0+fs/PUM3Gwpe6wBocZ6zwv0Ng5mSYNRl+Pqh1e9XEEovFX+8IAA2rtbrFzhtwghdCvCyEWC+E+EUd95uEECeEECtdH4MDD7NltOU8z/N9EqjEggMDkiTKuZH3mM43oQ5PUVpMlQEeGgHbUkET4DTAt2nw0HDXiF4Jqu7dj2Ew+PYUNpsdjBixrcXOG1CCF0JcBxillOOBTCFEHz8PGwK8LaWc6vrY3ZxAW8JMvvI7qyaBcm7h7RBEpCgXScAmWua95Mp0qDSC5pYBnAYoMcOGtIafH639ZlqKyaRx221vERdXicVShcVSjclkZ8qUVXTpktdy5w3weVOBxa7PlwMTgUNejxkLXCuEmADkAndKKR2EETtmv388EkE16r2qEhoa8J+u8G5Xfcu9DlXw4BGYHMRtQU8m6JuAeLMZ4FQCUBi8cwXT6dMd2bNnICAYOHAvmZlnQh1So3XunMePfvRXjhzpRXW1hR49jpGcrDcHstnMSAlWa12lnMAEmuATgZqXnRKgt5/HbAamSClPCyGeBa4APnF/gBBiPjBf/6prgKEEbimzMOL7tqmK+NqLsEr4iJXa+6s94P3Oel93gO/i4XcDIGEPjApS08Je5RDv8E3yZg16NNCQLFQj95UrJ7N27UTXNEPYtGkMl1yykcsui4y+MAAmk5N+/XJqvy4uTuGjj67hxAk9/2Vm5nPNNR/Tvv35oJwv0Bp8GRDv+jypjuPsklKedn1+APAp40gpX5RSjpJSjoL0AEMJXDlJ3MJbVGPBjhEHRqqx8Bd+yEbGtXo8imITnsm9RrURXu0evPNMPKv3jDe6jW9MGnSsgtHByS1BVViYxtq1E92mGRqw2y1s3DiOgoLWzx3B4HQaePnle8jN7YqmGdE0I6dOZfHKK/dQXd3wgszGCDTBb4XaieJDgeN+HvOmEGKoEMIIXAvsDPBcLcaMjZ/wBxyYMOPEgBMnRiprX7sUpXUVm+uuuecF8dfSIuG5bTCtAOIckOCAWWfgnzvCc2rdwYP9kNL3HZzTaeDAgX4hiKj5Dh7sS1VVHFK6v5obsNtN7NkzKCjnCLRE8zGwRgiRCcwGbhZC/FZK6T6j5kngLfQJ5Z9IKZc1K9IWMJfFDGUXia7ukgYggUp+xZO8wr2cpUNoA1RiTls7GOvI8A2VTtxpNJyo29rhZwcaf8xQMhicCOH7gxFCYjSG6Y7XDSgqaovD4bv6wG63cv5826CcI6AEL6UsEUJMBWYAf5JSnsFrhC6l3IM+kyZsXcPHJOH7V2PDzCTW8CHX+3mWpAfHcGDiZAiuG8SiWKm9A5gkzDsOr/fwLNNYnXDvsfqfqwFvdYV3u0CZCbpWwMOHYXSQ6vahnDWTnb2fZcsu87ndYJBkZ+8NQUTNl5FxBpPJic3mmYYtluqgXTwO+N2YlLJISrnYldwj0jna46jjR1BMG5/bRrOJI/RiD4M4SD92MoS+HGzpMJUYc9MpePgQdKzUL3r2K4E/7IJBDeyn+lJPWNQNysyAgBOJ8MtBsDulVcJuUSkppVx11WeYTHbMZhtmsw2Tyc7s2Uto27Y41OEFpEePY7Rvfw6j8eLMGaPRQXJyKf37B+etlZAyPFZsCjFK0sodHIezjTVM9NiPVUNvZZBFHk63NzjtOMdRepLi1kbYieAc6XTlBDasrRl6zIil0XtzVBngmgn6xVgPEkYUwV93Nf8c4TDvvawsgZwcvR7ft+/B2mmGzSUlVFVZsVhsGOuqkbWA6mozK1dOZdeuIUgpyM7ex/Tpy4mPr2r0MRYsWLBVn6jiKyZ70dRoSxEahtqLWhLBOdpzGcs8kjvAbSzC5NVH3ogknkrm8CkfcEMrRR0bVGJvmkKLvrG2DwEnEpp37JZI7CdOdOHLL2fx3XcdiY+vZMKEtYwduwnRwH97UlIFI0ZsD2ose/Zk8+WXs6ioSMRodDJ69GamT/+mVRK91Wpn1qyvmTXr6xY5fswm+Azy+YSray+wAkgkRhwc8p3RSRdO+t0NyoKNTPJbNNZYohJ7YNrbwM8kE5DQM8w22s7P78Sbb96O3a4vJiwrS2b58ulUVCS2+l6nR4704L///V5tLJpmZPPm0a6FSMcRQtK79xGsVlurxhUsMZvg5/EGBpwetxkAMw7m8Cnvc6PHfeuYyP/wIsmUedzuwMRGxrZ0uDFBJffAWTWYexLe6+J1cVaDu44HdsyWKsmsXDkVu90z9djtFjZsGMekSWuwWFpvwbsei+eqdbvdwtato9i1azBCgJQGrrvuQwYMiLzrbVGd4K1U8ShPcxevIdD4git4j7lsZgwZnCYe3y5uJux0oMDn9k+ZQw59GMB+EtDrY+UksIopbGZMi38v0Ugl9OC6+zik2OHtrvp8+p7l8OBhGBBmu09+911H/M3vMBg0Skra0L59ISUlyaxZM4Fjx3qSklLChAnr6NWrgWlEASgqqms6osBuj6v96oMPrufRR/9OUlJFHY8PT1Gc4CVfMZNRbKktrfTlGR7hn2gYOE8q1ViwYvN6lmAVU3yO5sTEZNbwOH/jNhbhwMxL3M9zPNgq302kU8m85Qnghjz9I5y1b3+O4uI2eO+5oGkGkpNLKC5O5oUXHqC62oKmmTh3Lp2TJ7tw+eVfMnJkcOvvGRmnOXQokYYmFAoh2b8/m9GjI2srz3BctBYUl7KC4Wz3qJsLwIDEhJMOFGLBhub2S1ZGIh9wPXvxv4qsgkR+yy8ZwEEGs4dneARHI/Z4jWUrVgiV3CNQS86YmTp1JSaTZ1Mts9nGqFFbsFrtrFkziepqK5p2cfxpt1tcOyLp9aeiojbk5PSmsLARrS/rMW3aCsxm75KQ78VVp9OAzRZ5f+tRO4K/hG+J93NR1J2ediS5dCGX7rzIfN7i1tYIL6qphK7Up0uXPG6++V2WLJlNYWE7rNZqxo7dyJQpqwE4erQnmuZ/f6mCgvasXTuJnJy+GI1OnE4j3bsfZ+7cxQHV7jMyznDXXa/z9deXcfp0BnFxVZSWJnm8uIC+oKpPH++GueEvahN8HllUEu9zUdSbQB+ZT2F16wSmKAq9ex/lBz94FqdTYDBIj+mRycmlnD/fzuc5TqeRnTuHkpPTF4fDXLv93fHj3Vm6dBZz5nxe+1gpaXDKZY2srHzuuuuN2q8/++wKdu4cit1uBiRms4NRozbToUMQ+zW3kqhN8B9wPU/zWKN6cnhvvK0ERo3cI19rL2byN9d8woT15OdnesxuMRod9Ox5lF27hvrsa+pwmNm5cyhXXvk5mzaNYc2aSZSXJ9KuXSGzZn1F375NG3lfeeUXZGfvY/fuQQgBQ4bsonv3E4F9gyEWtTX4ChKZzGr2MRA7xjo79EngD/ykNUNTFKUeffseYvr0ZZjNNqzWKkwmOz16HOP66z/EZvO/EY/DYWLNmol88810ysuTAEFhYXsWL76Ro0e7N+n8QkDPnsf53vc+4+qrPwvr5F5VVf/GRDHRqqAzJ5jDZ/ycp8gkH8nF6/fvcBO38jbeV/SVplGj9/B2NBEOJOv93odfqHtkV9cIvrLSSmlpMm3bXvBzUbJl2O0mzp1rT2JiGSkpeqn1jTdu4+jRnnh/BxkZeRQWtsdm820Z0rnzSe6775XWCLnVlJUl8uGH15Cb2x2n01xnq4KYSPDuupDLIzxDIqWUk0g2B9hPf57lYY7Sq8XPH61Ugm895Ua9pXBcI7rkOgT8aiBsbasPYYTU2wT/fTuku80QriuxOxwGPvvsKnbvHozR6ERKwaRJa5g0aW2ja9zBdPZsOxYuvA+Hw4TTacJgcGAyOZk7dzHvvHOzT/kGID6+giee+HPrB9tCpIRnn32Q8+fTXBejhepFU+Mk3Xiax9jKSJIpIYEqpvMN83mJ2SxhLZNCHWJEUYm99RxJhD/210fjAhhZBD8+AGn1bOP5fmc9ubs3Ias2wFMD4O+N2IJn6dJZ7NkzEKdTT6gAa9ZMIiWlhGHDgtDBrInS0wt56KHn+PbbMeTlZdKp03dccsm3pKSUYjQ6/Sb4du3CdIPZAJ040ZWSkpQ6Zxq5i7kED/AUPyONQsyuVgUW7Fiw8zL30I8cVLlGCTfFZnhkuD56r/n13NIWHh0Or22qu+TySaZvh0nNAPvaQKkJkuuptjgcRrZvH+6TNO12C2vWTApagt+1axCrVk2hpCSF9PSzzJjxNT165Nb5+JSUUmbM+Mbn9kmT1rBq1RSPi7Mmk43p05cHJc5wceFCaqMfG7UXWeszmyW1yd1dN07Qzms7+SHs5Bo+ogdHWys8JcaUmGBnG8iPq/sxSzqCXeAx9nAa4JwFdqTW/Tx7PX/hNffVVZ6x2Sx+t8kDKC8PzsyzLVtG8OmncygsbI/dbiE/P4v//Oc2jh/v1uRjTZiwnhkzviYpqRQhNNLTC7jppsX06HE8KLGGi4yMfDStcak7JkfwpSTTAf9zWmv2Y23DBZYwmyHswo4JKzb+y9Xczn98WgkrSiAksLCHXkYxSz2BDyyBJ/dAktf440Qi2Py8I5fA6XpeGCadhU8zweGVDzpVQVoDDRLj4yuJj6+grMx7xxBJ586n6n2upkFOTl/27x+A1VrN8OHbycj4zucx33wz3afZl8NhZtmy6bUXRisq4igqSiM1tYjExLoXLwoBY8ZsYcyYyGon0FQdOpyjd+/DHD7cC4ej/lk0MZmpnuVBfssva5uGAVRj4QuuoMI1J/5F5jOCbR69aubwGT/iz/yRn7Z6zOFG1d6b7+uO8GFnPXHX/JbtSYHfD4Cn9ng+dkAJrEiHKu+/WAG961nLd+dx2NgOLpih0gQWp36B9qf7G57zLgRcccUSPvro2tokLISG2Wznssvq3mJZ0wRvv30zubndsNmsCKGxbdtwZs782iP5VlXF1Tnt8ezZdDRN8MUXs9mxY1jtqtVBg/YwZ86nrbopRzi68cb3Wb9+HFu2jKS4ng2tYq5EM4Ov+Al/wowDCTgxUE482xnOPegjBitVfI//+jQiS6SCh3guBFEr0ehdr9a+AHYjbE7T91R1d9l3er3cfX9pixOyi6FfPQm+jQNe2azvzTrzDNyeC4s2Nb7DZHb2AW677S169jxCamoR2dl7uf/+hXTq5NtxtcbBg/1qkzvo7XYdDgtffTWTior42sdZrdUYjb6lUoDU1AusXTuBnTv1hU3V1XE4HGb27BnIihWXNi74KGY0akyatI7HHnum3sfF1Ai+O8f4iGs9NvkASQkpTGAdGvpfm5VqDPifg5bUQOsDRalPoQW+6aDX3Qvr6F1lkFBmhCS3C6DxGrywVS/prEsHkwZXntYTdkPiNLjijP4RiO7dc+nevREnctm7N9vvfHSDwcmxYz0YOHAfoK9inTBhHWvXTvQo05jNNi69dAWffHK1n/KNhU2bxnDZZdF14bSlhM0IfiB7WcVkvsfHLXaO+3kJE55zyoxIEqjgUvSdZOKo5Eo+5zy+XeocGFnC7BaLT4luG9Pgtkv0JP2fms2x/VQaEpzQwXerAtLs8OMc+O86+GAD3HMcLGFYqbBaqxHCd4AkBJjNnn9/Y8Zsonv3oxgMDoTQSEws4aqrPqV//xyqqvxfXLDZLGiNWAOghNEIPo4qJrOGEWzjjzzBb/ll0M/RjVys+E4aNiDJ4DS9OMw6JhBPJQmUI9H//gxABXGUk8RP+EPQ41Kin03Ab7I9pyw6BSD1sovTAELTE/bjB1t25NXS/WaGD9/Brl1DfEbfQkh69rw4G+3cuTRefvleHA4TmmbCbNY3vO7ZU9/YIzMzj1Onuvocv2PH7zCEzdA0vIXdjymJcn7G72jDhaAdsydHeJJf0pEzVOI7KjDhYD3jWcTttOccKZRicnWKd2LkMD35Lb+gPwc4ie8vXKxRF1ibbnebOlZXCH203r8ELj0L/9gOEyN4XU5FRRwrV07G6TQCEiGcmM1VWK1V3HrrW5hMF2vun3xyNZWVcbUvBHa7hbKyJL76agYAs2cvxWy2IYT+HP0Cr40rrljS6t9XpAqbEby7aqwMZzsraf7FlBt5l9e4GxMOLNhxInBiwOiqsZeRyDvczHnSGM722ttrmHESRzW/5+cATGcZP+ZPZJHHMi7jjzzBaTKbHacS3fR051+Pct9ZMy2lJUfvdruRp59+zNVm17XbgjRgNjt59NGnsVguJneHw8jJk13wHmNqmpGDB/sDehvf+fNfYs2aiZw+nUHHjt8xadJaOnas+wKv4iksE7wZO6fJaPZxEijnFe712NXJiKQaE/lkkk8mz/IQi7id1HreMQjXn+b9/Junebz2Im1vDnErbzGMHeST1ex4leg1qFifnugtzgmzT7d+PC1h+fJpHsldJ6ioSODQob4MHLj/4q1CIoTEXyss95k16ennuO66j1ss5mgXdiUaG2b2MJCD9G/2sSaxBie+q0Os2NnBUMbyLW9yBxIDRaSxkyE4vd5IV2LlTW7HShV/4X89ZuBYsdOGYp7gj82OVYluJqkvYIp3QJxDnwVjdcK072BCACUZp9DLPrtTXLX8VlZensDGjZfw9dfTOXSol2thUx/qavNx4EA/j6+NRo0+fQ5hMDi9brczZEgjmuQojRI2I3gnBiqxsInR3Mj7QTlmNb5TtWrUrFh1N49FrGUicVSRSBllJHGE3jzFL+jHQb/HsWBnJl8FJd5wp2rvzTOsGBZvgNXp+jz3UUXQs7zpx9meCgsG6p0iQX+x+PVe/fh1CWZpJje3C4sW3eaa325m8+bRZGScJj6+AjyacV+Umuob3Jw5n/Lqq3dTWpqMphkwGDQ6dChg+vQVQYs11oVNgj9If/qwlDw6B+2Ya5iE3c+m2GUkspD7fW7PoR/dOc4NvE93jrOFUXzJ5WgYKaADFvyv7c5TNXilkZKcgc9HB73p2M8G+y6Q+ukQ/cWjvuZhwaBp8N57N2K3Xxw82WxW8vMzGTZsq99ZLyCZOHGNz61JSRU89NCzHDvWk8LCNDp1+o4uXU6GpA1xtAqbBF9JfFCTO4ATE3P4lC+5HIHEgBMjGs/zfb5mht/nVJDIG9zpc/sZMljJVC5lhccK1zIS+DM/DmrcilKXFen+L9ZKYGU6zGnhen5BQUe/7QXsdgunTnVlzJiNbNo01uO+UaO2YLdbsFp9X30MBujV6yi9eqlmfi0hbBJ8S9nIODI4zVV8RhuK+YbpHKNnQMe6mXd4l5uYzGpsmDEg+Sm/ZymXBzlqRfGv1Aw2P1fOHAJK/KyMDfasGSG0OjtMCqGRk9Mfg0Fz61Uu2bJlFNu3D2fGjGWMHbspqPEo9Yv6BA9QSQLvMbf26wHs42f8juFsZxeD+R0/Zw+DGzxOMalczlIyyaMDBRygP1V+avmKUsMpYEM72NxW7954+Rno6GeVamMNL4K3uvqWaMwSRhQ1L9bG6NDhLImJ5Vy44Dlbxmy2kZl5hl27hnhtRKH3OHY6DSxbdhnduuX6dJVUWk5MJHh3o9nEci4ljipMaPTnAHP4lMtZyjomNuoY+WTF1LRIdXE1MDYBjw/Td2CqNIHZCW93hSf3wpjzgR1zYAmMLoTN7S4m+TgHjD0P/b0aiLXEnHch4KabFvP663egaQacTiMGg0avXodJTCyvszskgNNpZMeOYWRkLA16XIp/MZfgn+H/keQ21dGIRhIV/IuHGc6O0AWmRJ0vMuBw0sX2BHbXv78dAB+u16dONpUAfrUPVnaAJZ30r2efgakFrbcPWUbGGR5//GkOHOhPWVki3brlkpV1mj17srFYbH4bjYG+6Mlmq6PDmtIiAk7wQoiXgQHAF1LK3wb6mNY2kq1+bx/CLgQaMvyWBigRallH3+3yQC/b5CRBdiNb9nozAtML9I9QsVjsDBmy2+O2/v0P8PXXM7DbTUjp+42bzdVkZx9okXjOn2/L559fwdGjPTEanQwevJtZs5YSF9fAriZRLqBsJoS4DjBKKccDmUKIPoE8JhQukOr39lKS603uXcllHm9wBZ/7dKRUFH/MdXQ8lNTfBbLaACcSfHvChzuTSeO++xYyYMABVzfJmnZ9enLv3fsIvXodDvp5KyvjeOml+zh6tGft3Pxdu4bw5pvz/K6UjSWB/gpNBRa7Pl8OTAQOBfCYVvc0j/JzfuexIrWcBP7Fw3U8Q/JnfsRDPIcDExoGqrEynW8adWFWiV1z8uFAstcuTBJSHNDLz7YCEni7C7zZHYTUZ8ZML4DHc/SLqI3V0t0i65OcXM7cue8jJZw+3YkdO4Zit1vIzt5Pr16HA+4CWVERz9692VRWxtOz51GysvJr58tv3z4Mh8OElBcP7nSaKCjowKlTWXTpkheE7ywyBZrgE4Gan1oJ0DuQxwgh5gPz9a9ap0vjH/kJncnjbl6hGitWbLzFLfyKX/t9/Bw+5QH+Tbzb9n4a8DlX0p3jqqSj1OnSs7C1rV6qEei9aEwSntrtv16+rIOe3N1nyKzoABYNHgv50KhphIDMzDNkZjZjVZfLsWPdeeutWwC9SdmaNZPo2zeH66//AIMBzpzp5NOaWCc5dy49phN8oNmpDGrnBybVcZwGHyOlfFFKOUpKOQrSAwylaTSMPMRzZJHPZXxDZ04xn4V1bqT9AC+QhOd6cgPQliJGsK0VIg4tNYPmokoDLE+HzzPgTD0bXdcQwP/mwEtb4aHD8MQBeG899KqjPcF/uvlOf6w2wped/M99jwVOp4F3352L3W7BbrcgpRG73UJOTl/27csGICPjNCaT/1p7evrZ1gw37AQ6gt+KXnLZCAwFv41aGvOYoDNjI4s8zpJOOUl1Pq6INLb62bXJWyL+/xo1DCR4bP2nRLPdbeAng11VZaG/i5t7Eu493vBzu1boHw05X8cMQymgwqiP5OsTytJMSzl5sjOa5jvIsNst7NgxjEGD9jFs2E5Wr56M03mxTGM02unYsYCsrNgdvUPgI/iPgXlCiL8Bc4G9QgjvWTLej/k80CAb62Ge4Szp7GYwZ0nneR5o9gXRt7mZchJ8bhdINjGmWcdWIoNNwM8GQYVJn89eZQSbEd7vAjtSg3ee7BK99u4t2Q4pXr/GEvg0A24aCzMmw/yRcKzbseAFE0Hi46u4//6F9Ox5BIPBidlsY+jQXcybtyjm+9oENIKXUpYIIaYCM4A/SSnPADsbeEw9ve6a7wbe4w/81OPi6TzewI6Z/8c/ATBiYyTbOEhfihsxegd4lXu4kzcYxG6SKceGGQcmVw2/Ee/TlYi3IxX8DCKpNsAXnWDYBX1Ev6UtHE+ELhX6QiY/MyTrNf8o7GqjH1dzDb2sTnj4sO9I7J0u8Eb3iyWdQ8lw/Na3mPfmPLr6bfgVmbp0OYXB4PuqZzZXM2zYxZSTllbEvHlvtWZoESHgyp6UskhKudiV3AN+TLD8gt94JHeARCq5l5exUsUyLsWOlY2Mo4h25NOJePxMZfBiw8ok1nAPr/I6d/A0jzKUnXzIDS31rShhxl7HX4kUejIuNcG9o/UWvi/11PdevWsMXGjimp6e5fD8NphyDjpV6q0H/rBbv1jrziFgkZ96vd1iZ/m05R63VVTEcfRoDwoKWucaV7AZjRpz5y7GbLa5tu/Tt+3r2/cQ2dl7Qx1e2IuwmbZ1yyLf7+0CyX+4lWms9Ji50InvOEpPMmh4tYgTE+9zI+9zY5CiDX/q4upFwy7431QjzgHTCuC5XnAqHhyuFwKHAU4b4O99YMG+pp2rWwX8n9dz9ifD0k56qWjqWeheXvcmHwUdLv4+r1gxmXXrJmI0OtE0A+3bn+O2294iKSmAJvQh1LPncR577O/s3TvQbZpkXsyXXxoj4hN8W85zC29TTBvSKPR5S1JKMtfwsc+0NAF05CydyOMMWYBkFFsYxB5y6Mt6xtN6i7+VcJbohMcPwt/66aNnp9C32htxASaegz8MuJjcazgNsK59XdtfNN6irvpo3Sb0dwwrOsC4enaAalfYDoD9+/uzfv0EHA4zDof+VuK77zry7rs3cu+9rzUjotBISKhk9OgtoQ4j4kR0gh/FZpYxHRMOEqmsXTdX8wdVTgI/5K+8wR11HmMoOykhlS+5nOFsB0AiOEQfprGc4jpWviqxZWYBDCiFrzrqLXsnnIORRXqNs67JLZpoXoIvsMKb3fQLujWqTHp3yklnYW26Z5nGZDNx6Up9o/oNG8b6zA3XNCOnT2dSXJyCEBpLlswmJ6cvAAMG7Gf27CUkJlaiRI8Inl0reZebaEMpia5NtQX61n/FpLCO8VzPByxiHjbMde5ov5rJ/I6fMorNJFFOEuUkU8ZA9vIsD7Xad6OEvy6V+rTIRw/B6KKLfzzjC8HoleUNGow+37w/sE1p/p9fbdRn1tyeC0l2feZN6vm2TH7h+1jXTkDToKLCd+YXgMGgUVqayMKF93HgQD+cThNOp4l9+wbwyiv3+J2SqESuiB3B9+IIHfHtK21CI49MJrIOgH4cIIe+DMLzgowEtjCCSpK4gzeIx7NJtxUbN/A+83hTrVZV6vXwYdiboveOqTTpG2vHO+GxnOYd1+r0P23SICFeg9tOwK0nYMWa7/HH377LOiFZKwVxcVX06HGUoqK2OJ2ef+IGg8a5c+lUVcV5NATTNBOlpckcOtSbfv0ibNmsUqeITfBOjIg6xuU1q1L7s59NjCGBitq3yTXP+IwruZrPADy24HNnwoEBDadK8Eo92tlg0SZYla73fu9WobfvjWtgYVJDxhfC3/wMqE0SZrnGNh99+AzPP/9AbZ0dwGazsGfPYJxOIzVFIiE0TCYHV1zxuWvbPd+Wvg6HiXPn0oOW4J1OwcaNY9m8eRR2u5n+/Q9y6aUrI+4ibySL2AR/nB4cpzv9OYDBLdGXk8BC7gPgN/ySBMoxut0vgPOkch0fMosvmchaDtKXwezG5FZNdSJYy8Q6WxhEKzV7JjAWDWYEeaOiRCf8Zg/830D991aiX+B9+PDFlbHbtg1H07wHIMKV3C8Oa4TQmDv3Xfr0OcrOnQYslmqfJG8yOWjfPnhL+z/44DpycvricOjXArZvH0ZOTl8eeujZmG/j21oiOnvdwPusYgpWqrFSjR0za5jEczwIwDg2eCT3GlZsrGM8AzhIEmVUEocBSSVxxFNFOfHYsDKfF1v7W1LCVIlJX8hkknptPb6Zo/PGGlUEH6yHzWn6TJ2R56GN297VZWVJXlvk1RAenxsMkvz8TM6e7cC6deNdOy9dvARsMDhISiqjT5/gtPM9d64dOTn9PN5ZaJqJqqo4duwYytixm9E0OHasBxcupJKZeZqMjBZfLhNzIjrB7yebrpzgGj6mCyeIp5LOnOIJ/sCr3MMpsvzOj7dgYxB7SXB1iKz5t4J4FjOXXQzmNe7mPO1a9ftRwtOSjvD3vno3SIE+a+bXe2FMK+yBCvqLyeRz/u/r2/cQ+/cPqHMXpRo1PdJLStp4za6RGAwa2dn6LBp/q0brYrOZ+eabaezYMRRNM9K792Euv3wpbdqUcPp0BgaDE/Bc7WW3W8jN7UZ29n5effVuysoS0TQjmmbAbLYxceJaxo79FovF4f+kSpMIGSYd8YUYJSGwea5JlLKBcXQjl2TKcGDEiBPNteGv0a30UkE8VVhJ44LPccpJYATbyKFfgN9F5FMlGk8n4+H+Ub47M8U59c6QSc7QxFXD6TTyyCOryMkZ7Za4fSdnmkw2VyL1Hu1r9O17kFtvXUxTSAmvvnoXeXmZOJ16EhdCIyGhgh/84J8UFHRg0aLbfV54jEYH48at59SpzuTmdvPZ+clgcNKhQwH33bcQk6mV3iZFuAULFmzVO/L6ioqrhz/kL/TiCMmu1gMmnHr/baRrGz69pu7EwDKmc5LOdR7L2eQOIko0W9ZRX9zkTUh9IVOoGY1O/va3acyc+RXduh2nd+8cOnQ4g9F4sTuZfoHVjsnkb1RsoKCgU5PPm5eXyenTGbXJHWr2XLWwc+dQunQ5RWrqBdco3jPeIUN2c/JkV7/b+mmakfPn09i/f4DrmHDiRBdWrpzMt9+Oprw83uc5St0iukRT42be9diQw50BfTxjROJEMp3lLOI2enPUo3eNBuSRyRF6tUrM4ShSR+/HEuGZ3npL3zgNrsyHe4/Vvy1eY1Ua/Tca04RvL5hQsVhsjB69ldGj9f2G7XYTK1ZMZfv24TgcJvr0OcTUqSt48cX/8fNsGVDP9IKCDn5vt9st5OdnIgTccccbfPTRtRw/3h2ANm0ucO21/yUxsf5ZNDablSNHejFw4D4WL76RI0d6YbebMJkcLFt2Gbfe+jY9ehxvcsyxKCoSfFUDXR1r/j6NQCIVXMWnrGYSk1iLBRtVWLFj4Xo+RLUniCxnrfDwcL1fOgLKDfBxFuTFw28D7EVVaYBPMmF1up7ITRrYvZK5BC4539zog2fKlAW1/eDNZgczZy5j5sxlHo8ZMWIr27eP8KjBm812pkxZ3eTztWtXiPAzSd9kstGxoz6dKCmpgnnz/kNVlQWHw0RiYkVt/5i0tPOcPev/RcJodJCSUsyePYNcyV2Pt2Y2zuLFN/KjH/0Vo/fqMsVHVJRonucBvz3b69KWYu5jITP5ip/zFA/yPF04qfZYjUAfZLl2O3J7XbYZ9Vkn+QF0c642wEMj4NUesK8NHEjRk7zRCUi9NGN1wi0noJP/N41h6/LLv2LcuPWYzTZAkppaxM03v0Pnzk3fFKNr15OkpZ3HYHAv+2iYTE6GDdvh8di4OBtJSRUezcGuvfYjzOZq8DPLzWDQGDFiR+1+rt40zcCpU1lNjjkWRUWCX8j9fMLVVBCPw+/ESE8GNEpowwbG8xf+l/9wO5VNeIFQwkdOsm+jL9A3qc4N4L90WUc4Hed5UdVp0PcYnVoAlxTCwGK9LLS2fd19aMJRRUU8+/YNRAiJ2WynvDyRdesmYPd+e9IIQsCdd75BdvZ+DAYnQmh063aCe+99mYSEhl/5MjPP8MgjzzB27Hqs1iqMRgcWSzUJCeXcdNO7pKbWvX2ElEJ1kmykqCjRaBi5lbfJZi+TWM1MvuIylpFEeW39vUYVVj7haspIDl3AStD0LtVr795J3i6gawB9szak6Q29vFk0vR6/s+3F2vumdjC2UG/vGw75ZsqUBUDdW/d98snVnD+f5jGT5sSJrqxePZnp01c0+Xzx8VXccMOHaNqHSCkwGpt20SMpqYLLL1/GrFnLKCxsh91uomPHgtqpmsOH7+DkyS4+o3ij0UFW1qkmxxuLomIEX2MfA/k33+d6PiKLfN5gHtK1ztWBkSosrGQq9/JyqENVguT6PNdepW65xeLUN8vIakSC35AGd4yBaVPghnFQYtYbhXlzAtvSPC+sVhlhYzv9BSacTJmyoDbZ17DbTRw+3NtnmqTDYWb79uHNOp/BQJOTuzshoH37QjIyvvOYhz9w4B769s3BbLbVbsVnsVRz882Lm3W+WBIVI3h/vuAKRrMZE/o0LQNO7MRxLy+r0buXSJ09A9CxGv6xHf7RB/a2AasGs0/DA0cbfu6WtvDrgRfLMYVWfXcm7xbAQuozcqr85JQqA3ybBkNadEPKwEyasoCXztzMziE7kQ4jcvtQ+OJKvN9vOBzhmQYMBrjxxg/Iy8vk6NEeJCRUkp29j/j4CLv4EULh+T/bTEPYyQi2EefWIdIAmHHwfZ7nl3jvD65Est7l8M8dTe+9vrCH7wImW01/LrdknuiAm0/ovdm9t3A3a5AUhosuJfC7AbBu4ju1JSex+CZ4Yx48+ELt4wwGJ/36HQxNkI2UlZVPVpb/HduU+kVViaZGHw7h8PPaFUc1g9kVgoiU1tDU9yEn67sIKy5+2A2uJO7nBAZgesO7Pra6fSn6Qiz36wkysQLufAPjsG0AmM02EhPLueyyZXUcRYl0UTmC38MgzD5jLaggjk1cEoKIlGCoMsDKDnA8Qd+gespZvSQTqKxKOOSvWueVyKuN8FkmPLUb/m/QxcG9JuCn+6FDtc8RmqzSqPd5b873425Tmj7l05vJWkmPB14g8YXv07nzKQYP3o3V6vu3okSHqEzwB+nPN0xjOstJcO325MRAJQn8G3+r+ZRwV2CFB0foC5pqNtVY2AOe2wbtA+w8e99RPWF7lGnqqPOUmfQ9WD9cBztS9eQ+7ELze74fS4Q/9oPDSfppRxbBjw9AWjNzbpLD/wItk4T+mWcYec0nzTuBEhGiskQDcAMf8A8e4RztqCCeT7mKMWyikDBoIBImVqwQEXOB9e99oMiiJ3fQ/z1vgX/1DvyYY4rgl/sgq0K/kNq2Wm8i5s2k6Xuwgn6xdUwRjD3f/ORebIIfDNfn8jsN+lTPLW3h0eHNn18/rcD/H7cUcF/bT31m2SjRKSpH8AA2rPyM3/Mzfh/qUJRmkuglB++eME4DrG9mR+cJhfpHzcB9Qzv4dbY+j14z6KtWkx369njB9lUnvZGZdPu+nAY4Z4HtqTDyQuDHbmfT5+f/ZoBnov/1XkhxXRRuaN68EvmiNsEr0aWu9xnBegtac/xxhfD8NvgwS1/ROrII5pxu3kwZm9B746Taoa1b6SU3wXcWD+gvZKfjwU9H6yYZXwgfr9dLSkLqJaVgNGBTIodK8ErYE8Ckc7C6vT7CrWHS9AutwdajHH7YzA2za3ySAf/upb9DcBhg1Hn4xX5IcMKAEvimg+/KWQH0KgvO+a3axaZopSZY2E3fO9aswZx8faGYEr2itgavRJf/dwgyq/SLqyZN/zerUt+fNFxtagvP94YKk37NwO6qsf9Wb3XO9AK9/OPeFNHihH4l0L80uLFUG+CBEXqnzYI4yEvQG6otGIiqx0cxNYJXIkKqHV7dpHeJPJEA3cv1/UrDeYTydlffnvF2I2xJgyKzXq55YSu81FNvXGbWYPYZuCM3+L1tlnfQL0rb3X5g1Ub9BedoYpBPpoQNleBjVKTMnnFnRJ+9MjaM+rDX52wd26SaNbhg0RN8mh2eOKh/tKRdbfw3URPAgWRQOT46hfMASFEi2vALnuWXGhJ9amZryqzUyz/ehAzOQi0lPEVUgu/Ad7zKXRSRylna8xceJ4H6t/9SlFC5PRfinZ5J3uqE+UdbfzbLFWfAuwGjQYM2dhhe1LqxKK0nYko0cVSyiTFkkI8Ffc7agzzHeDYwnvWER0duRbmoYzUs3AKLuunz2ttXw60nYUwISkztbPCXnXoDsrNWfe59/xJ9Rk9ztpa9cCEFKQWpqcVqE44wFDYJfiRbeY1BPMI/WM50n/tv4l3SKKxN7gDxVDOI3UxkLWuZ1JrhRqxIrL1Hso7VwZty2VzZpfDmJn0hlUXqo3cIbKFTQUE67713A0VFbQFISSnhxhvfJyPjTBAjVpqrySUaIcTLQoj1Qohf1PMYkxDihBBipeujUZudDmIvn3A1o9nkc98otpDspxxjRGOI6hCpKI0igHTbxeQeCJvNxKuv3sXZs+1xOMw4HGbOn0/jtdfupKqqjivLSkg0KcELIa4DjFLK8UCmEKJPHQ8dArwtpZzq+tjd2HPEU8kv+Y3P7fvp73djbQcmDtOMhiSKEqNWrVpQ7+hd1nGdYP/+ATidRjzTh0DTDOzZMzCYISrN1NQSzVRgsevz5cBE4JCfx40FrhVCTABygTullD6LvYUQ84H5AF1dtxmQDGSvzwEXMY8n+RVxVNbusWrDxBk6sozLmvhtKErDzpvhcDKkV+urW1uTE1iSAZ9m6q0OphfA9acgvpldyBpTjjl1KosvvphNfn4mFouNUaO2MG3ackwm/eSlpcl+d4Gy282UlKQ0L0AlqOpN8EKIfwP93G6aArUbmpZAnUPnzcAUKeVpIcSzwBWAT39SKeWLwIsAo4SQAE4EOxnic8AS2jCe9SzkPsaxAQ0DXzKL+1mI1qzLRIriSQLP9YL/Zur7vToMeoL/w+7mlTaa4ncDYH37iwul3oyHlenwwja95W9TNbbOfu5cO15//Y7aja5tNiubNo2mpCSZG274CIDOnU9hNDp89ne1WGx06aI2ww4n9SZ4KaVH83QhxD+AeNeXSdRd4tklpayZXXsAqKuU46OKeJ7kV37vy6Efk1lDHJVoGLBhBSSTWE1PjrKNEez28+KgKE2xtKO+wYfdeLGf+qEkvcXAn1vhcs+xBH03JvdGZDYj5Mfr/XimBaH/jqaBzWbBYrFhcPsrXrduvM/o3OGwcODAAEpLvyY5uYxu3U6QlZXPqVNZOBz6C4HJZCc9/Sy9eoVx74gY1NQSzVb0ssxGYChQ1/q7N4UQTwF7gGuB3zV0YAlsYSSP8nd2UP8u71Wu15h0CljBpXTlBCAxIFnJFK7lY+xYGvs9xQQ1e0ZXbIJVHfTGWyOKYICfni/vd/ZtMeA0wM5UuGDW2ya0pL1t9AVI3ipNsL1t4xN8XaP2JUtmsmnTJUhXn+LevQ9x661vYzDAmTOdkNJ33GY0Ojh/Po3k5DKEgNtvX8S3317C9u3DkVIwdOhOxo3b6PFioYReUxP8x8AaIUQmMBsYK4TIBm6VUrrPqnkSeAv9ov0nUsoGN33cxkhGs6VJwbzC3fThEBa37fkuZSU/4ff8po53AUrs2pYKP3fN57IJfX76+HPw8/2eb0VLzf6fb5RQbmz5BN/O5v+tsdkJHarqfl5jyjArVkzh22/H4r5u5PDhPrz55u3ceeciMjLyOXOmI1J6vsI5HGbS0gprvzaZNCZM2MCECRsaPKcSOk1K8FLKEiHEVGAG8CcpZTFQDPzC63F7oGVrJQmUM5OvPZK7fnsl83lJJXjFg0PonRPdR+ZV6BuGrE6HqW6j4nHn9BKN0yvLJjggo54EGyyjz+s7S1UaPTcDMQKXe00zd0/q1dUW9u3LpqwskW7dTtCly0mfxUfr1k3Ad1Gg4NixntjtBiZMWM+ePYOx2S7+oEwmGwMH7iM5Wa0ajzRNXugkpSzi4kyakPG3qXaNOFrhr1CJKHtSwOmnSlVlgi87eSb4O07AmnR9H1abUV/Sb5bwo4Ot09vDJOHvO/T9Yk/H6eWaRKe+6jTd5n+knp+fweuv34GmCRwOEyaTk27dcrnllncwuvVK8Df7pUZJSRvatSvirrteY8mS2eTlZWG1VjN69GamTFnVAt+p0tLCZiVrUxWTyn4GMJhdHn90Nkx8zPdCFpcSpuq5BOFd7k6zwSub4ZNM2NZWb9R1fR70bMUBbJdKeHUz5MXpLX67VsCaVQvwl2alhHffnUt1dVztbXa7kdzcbmzbNpzRo7fW3m4227Db/S9GatOmGIDMzDPce++rQf1+lNCI2AQPcDevspKpmLETTxVlJHKBVH7e8DXdmKEuruoGFoPBz4XLOKdv2QOgjQPmndA/Qunw0gWAvpikLmfPtqeiwncRoN1u8Unw06atYOnSWXi+4kmys/fWznNXokdEJ/jtjKAvOdzDy/TnAOsZzyLmUU5SqENTwoxZwoK98IvB+ojdbtDnuI8rbJlt/wIVSF8YKet7Efe8b9y4b9E0AytWXIrDYcJg0Bg2bDtXX/15k8+rhL+ITvAA39GJ3/PzUIehRICRF+CdjbA8HcrM+obaA0rCow9pIIm9RocOZ4mPr6xdnFTDbLYxbNgOn8er2S+xI+ITvKKcjoM17S9uzt2pnmvsbexwbb7v7TYBu9voxxhcrI/4W1pzkro7IWDu3Pd44415SCmw281YLDaysvIYOXJrwwdQopZK8EpEez9L39O0Jh8v7AEPHPGfxOuyqS08OfDiMQTw6736CL8lBCuxu+vcOY/HHnuaPXsGUV6eRNeuufTocVz1aI9xKsFHsWi/wJoXpyd3m9eq0xd66fu2CvSNNpIdMKbQ/y5KRWZ9OmK11zF+MQje2aBfbA2mlkjuNeLjqz0uqCqKSvBKxFqTDpqf1zAJ/LEf7E/RZ84Y0Feh/mUn9C3zfOyKDnUff1UHuLoJ7wQUJdyoBK9ErLrK5JrQ+7k4vFYlPTASupbrUx+nF+i3lZn0GTXe7EK/L1hacuSuKHVRrYGUiDXxnP+57ZrwTe6gL/vPTYK/9IPFnfXbRhXp0yW9mSSMCsLeqQ1tqqEoLUkleCVidamEu4+DxQlGDUya/nlmRf3PqzLCa931mTMDSvSGY3FutfY4hz433rucoyiRRpVolIh280l9JL86Xe/ZMvkc7GoDz/Txbfnr7WwcZFXq3STXtoelnfQLs5efgQnnmh9bOIzcz59PZd++gTidBvr1O0inTgWhDklpRSrBR6Fonz3jrXMl3OrWUqBjlb5pR06y3kPdH6eAVJv+uQH9hWFyEJI6hEdiB9iyZThffjkbTRNIaWD16klkZ+9jzpzPsVhaaWsqJaRUgleijknCX3fChvbwaQZsaQuaWzHS6oTp3+kdGqNVaWkiX345G4fjYnN7p9PA7t1D2L8/mxkzvuaSSzaHMEKlNagEr0QlI3rpZuI5WNYBnu0NFa6SzeVn4KEW2FkuXEbuADk5fRH+toVC4HCYWbbsMtq3L6RXr6OtHpvSelSCV6LeZQUwrUDfbi/J6X/WTHOFU3IH6kjuF9ntFjZsGKsSfJRTs2iUmGAA0uz1J3cNKLAGd/57qPTrl9NAl0koLU1upWiUUImCX2VFab4NafDXfvpm3FLoc+B/dgCSGmhVEG4j9xqJiRXMmfMJn3xyNU6nCe+emUajgz59DoUmOKXVqBG8EvMOJ8KvB0KhVe9rYzfA5rbw80Ghjqx5hg7dw6OPPsPAgXsxGh3UrP01Gh0kJFQwbpxqGRzt1Ag+isTa9Mhgea+L3prAncMIB5PhZLy+oCpSJSeXceONH3DsWHfWrx9LaWkKffocYuzYjSQmRvA3pjSKSvBKzMuL95xGWcMk4azVf4IP19JMXXr0OE6PHsdDHYbSylSJRol5wy6A2c+ceLto3Y22FSXYVIJXYt71pyDB1c+mRpwTvpcHqX4WfEba6F2JXapEo8S8tnZ4cSu81g02tYMUO9xwCmafCXVkitI8KsErCtChGn6cU/9j1MhdiTRRVaLpxnH6sx9BCyxVVBRFiTBRMYLvwVE+5Dr6koOGgVKSuZ1FLGd6qENTooAauSuRKuITvAEnK5lKFnkYXSP3JMr5L99jIHs5QbcQR6goihIaEV+imcZyUrlQm9xrmLFzLwtDFJWiKEroRXyCzyQf4Wf7ZSs2upMbgoiUaKLKM0oki/gEv4FxGPFdpVJKIsu4LAQRKYqihIeIT/CH6Mu7zKWMxNrbKonjBF1ZzNwQRhbbyo3weSdY1BW2p+LnPZaiKC0toIusQoiOwPtSykn1PMYMfASkAQullK8EFmLD7uUV1jCZB3mOBCp4l5v4G49TTVxLnVKpR04SPD5M3/e02jWEMEgYfAHmH4MBpaGMrnFUaUaJBk0ewQsh2gKvg9uQ2b8fAFuklOOBq4QQLba7gMTAq9zDaLYwkH08ya8oQ21mEAoSWDAQyk1QZdR7q0sBTgPsSIPHhsG+lFBHqSixIZASjRO4CShp4HFTgcWuz9cDowI4l9IEl14qufTS0BZDTsVDkaXu+6uN8O+erRePosSyBks0Qoh/A/3cbloupXxSiAZ7jycCea7PS4COfo49H5ivf9W14WiVqHA4KdQR1E2VZpRo0mCCl1L+T4DHLgPigWIgyfW197FfBF4EEGKUug4XJDWj+FBsANK5Etra4HR83Y9Jr269eBQllrXkLJqtwETX50OB4y14LiVMCGDBXkh0gEnDZ/qM1Ql3HA9BYA1YtWqBGr0rUScoCV4IMU0I8bDXza8DvxZC/APIBr4NxrmUxgtVPb5vGby7AR7JgcHF+mYaZick2eGBIzDtbEjCUpSYE3AvGinlVLfPlwPLve7PFULMQB/F/5+U0s+eOUq0SnTCVWf0D5uAMjO0sYNRFeIUpdW0aLMxKWU+F2fSKCEQynp8DYuENFvITt8gVZpRolXEr2RVFEVR/FMJXlEUJUpFfD94pXHCoVQTblRpRol2agSvKIoSpVSCVxRFiVKqRKPEHFWaUWKFSvCKEgSaBocP92HPnoGYTA6GD99Bly6nQh2WEuNUgldiSkuM3qWE99+/gUOHemO3WwGN3bsHM3HiWqZMWRP08ylKY6kavKI009GjPd2SO4ABu93C6tWTKC5Wze+V0FEJXlGa6cCBftjtvk3wDQbJkSO9QhCRouhUiUaJCS15YdVisSGEhpRGj9uFkJjNYdyjQYl6agSvKM00bNhOjEbN7319++a0cjSKcpFK8ErIVRrggyz44VB4agDsjbCydXr6OWbP/gKTyY7FUo3VWoXVWsUtt7yD1WoPdXhKDFMlGiWkKo3wwAj4Lk7fr1VIWNMeHj4MV50OdXSNN3LkDrKzD3DkSE9MJge9eh3FbHaEOiwlxqkEr4TUJxkXkzuAFPrn/+oN07+DeP+Vj0ZrzUVN8fFVDBq0r9XOpygNUSUaJaTWpF9M7u5MEg5GWKlGUcKNkDI8ttgRQpwFchv58PbAuRYMp6VFcvyRHDuo+EMtkuMP19i7SSnT/d0RNgm+KYQQW6SUo0IdR6AiOf5Ijh1U/KEWyfFHYuyqRKMoihKlVIJXFEWJUpGa4F8MdQDNFMnxR3LsoOIPtUiOP+Jij8gavKIoitKwiBzBCyHShBAzhBDtQx2LoijRL1JzTsQleCFEBvA5MAZYIYTwOz0oXAkh2gghlgghvhZCfCSE8G1DGOaEEB2FEKrReSuL1J97pP/OR3LOibgEDwwEHpNSPgUsBUaEOJ6mug34m5RyBnAGuDzE8TSJEKIt8DqQGOpYmkoI8bIQYr0Q4hehjqWpIvnnToT/zhPBOSfiEryUcpmUcqMQYjL6K+qGUMfUFFLK56SUX7u+TAcKQhlPAJzATUBJqANpCiHEdYBRSjkeyBRC9Al1TE0UkT93iPzf+UjOOWHfi0YI8W+gn9tNy4HfoP+y29F/8cOWv/illE8KIcYBbaWUG0MUWqPUE3+oQgrUVGCx6/PlwETgUMiiaSIpZQlABP7ca0XK77w/Qv/BR0TOcRf2CV5K+T913PWQEOI3wFXAu60YUpP4i18IkQb8E7i+9SNqmnp+/pEmEchzfV4C9A5hLDEnkn7n/ZH6dMOIyDnuIq5EI4R4Qghxh+vLVOBC6KJpOtcFpsXAT6WUje29ozRfGRDv+jyJCPzdj1SR/jsfyTknEn/JXwTmCSFWA0bgqxDH01T3AiOBnwshVgohbgp1QDFiK3pZBmAocDx0ocScSP+dj9icoxY6KTFBCJECrAG+AWYDY6WUxaGNSlFalkrwSsxwTTWcAayWUp4JdTyK0tJUglcURYlSkViDVxRFURpBJXhFUZQopRK8oihKlFIJXlEUJUqpBK8oihKl/j8X+LOI1lRM6wAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mpl\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "# 定义类别转换函数\n",
    "def trtype(s):\n",
    "    types = {b'Iris-setosa': 0, b'Iris-versicolor': 1, b'Iris-virginica': 2}\n",
    "    return types[s]\n",
    "\n",
    "\n",
    "# 划分测试集和训练集\n",
    "def label_tr(input_label):  # 标签转换，将一维标签转换为三维\n",
    "    label_list = {0: [1, 0, 0], 1: [0, 1, 0], 2: [0, 0, 1]}\n",
    "    label_change = []\n",
    "    for i in range(len(input_label)):\n",
    "        label_change.append(label_list[int(input_label[i])])\n",
    "    return np.array(label_change)\n",
    "\n",
    "\n",
    "def inv_label_tr(input_label_change):  # 标签转换逆过程\n",
    "    y_real = []\n",
    "    for i in range(input_label_change.shape[0]):\n",
    "        for j in range(3):\n",
    "            if input_label_change[i][j] == 1:\n",
    "                y_lable = j\n",
    "        y_real.append(y_lable)\n",
    "    return np.array(y_real)\n",
    "\n",
    "\n",
    "def rand(a, b):  # 随机数函数 ---> 生成 a - b 之间的随机数\n",
    "    return (b - a) * random.random() + a   # random.random()用于生成随机的 0 - 1之间的浮点数\n",
    "\n",
    "\n",
    "def make_matrix(m, n, fill=0.0):  # 矩阵生成函数\n",
    "    mat = []\n",
    "    for i in range(m):\n",
    "        mat.append([fill] * n)\n",
    "    return mat    # 生成 m x n 的零矩阵\n",
    "\n",
    "\n",
    "def sigmoid(x):  # 激活函数\n",
    "    return 1.0 / (1.0 + math.exp(-x))\n",
    "\n",
    "\n",
    "def sigmoid_derivative(x):  # 激活函数求导\n",
    "    return x * (1 - x)\n",
    "\n",
    "\n",
    "class BPNeuralNetwork:  # BP神经网络类\n",
    "    def __init__(self):  # 初始化\n",
    "        self.input_n = 0   # 输入层\n",
    "        self.hidden_n = 0  # 隐含层\n",
    "        self.output_n = 0  # 输出层\n",
    "        self.input_cells = []  # 输入层节点数\n",
    "        self.hidden_cells = []  # 隐含层节点数\n",
    "        self.output_cells = []  # 输出层节点数\n",
    "        self.input_weights = []  # 输入层-->隐含层间的权重\n",
    "        self.output_weights = []  # 隐含层-->输出层间的权重\n",
    "        self.input_correction = []\n",
    "        self.output_correction = []\n",
    "\n",
    "        \n",
    "    def setup(self, n_in, n_hidden, n_out):   # 输入 2 5 3 \n",
    "        # 初始化输入、隐层、输出元数\n",
    "        self.input_n = n_in + 1  # n_in+1为输入层增加一个X0单元，代码中也加入了b值：a[1]=W.T*X+b\n",
    "        self.hidden_n = n_hidden\n",
    "        self.output_n = n_out\n",
    "        # 初始化神经元\n",
    "        self.input_cells = [1.0] * self.input_n   # 激活神经元\n",
    "        self.hidden_cells = [1.0] * self.hidden_n\n",
    "        self.output_cells = [1.0] * self.output_n\n",
    "        # 初始化权重矩阵\n",
    "        self.input_weights = make_matrix(self.input_n, self.hidden_n)  # 创建一个 input_n x hidden_n 的权重矩阵 3 X 5\n",
    "        self.output_weights = make_matrix(self.hidden_n, self.output_n)  # 5 X 3\n",
    "        # 初始化权重\n",
    "        for i in range(self.input_n):\n",
    "            for h in range(self.hidden_n):\n",
    "                self.input_weights[i][h] = rand(-0.2, 0.2)\n",
    "        for h in range(self.hidden_n):\n",
    "            for o in range(self.output_n):\n",
    "                self.output_weights[h][o] = rand(-2.0, 2.0)\n",
    "        # 初始化偏置\n",
    "        self.input_correction = make_matrix(self.input_n, self.hidden_n)\n",
    "        self.output_correction = make_matrix(self.hidden_n, self.output_n)\n",
    "\n",
    "        \n",
    "    def predict(self, inputs):   # inputs = x_train\n",
    "        #  激活输入层\n",
    "        for i in range(self.input_n - 1):  # i = 0 , 1 , 2\n",
    "            self.input_cells[i] = inputs[i]  # ----> input_cells = [x_test]\n",
    "        # 激活隐层\n",
    "        for j in range(self.hidden_n):\n",
    "            total = 0.0\n",
    "            for i in range(self.input_n):\n",
    "                total += self.input_cells[i] * self.input_weights[i][j]   # 神经元中是测试集的数据，乘以权重矩阵就是这一层的输入值\n",
    "            self.hidden_cells[j] = sigmoid(total)    # 隐层输出\n",
    "        # 激活输出层\n",
    "        for k in range(self.output_n):\n",
    "            total = 0.0\n",
    "            for j in range(self.hidden_n):\n",
    "                total += self.hidden_cells[j] * self.output_weights[j][k]  # 隐层输入 x 输入权重 = 输出层输入\n",
    "            self.output_cells[k] = sigmoid(total)    # 输出层输出\n",
    "        return self.output_cells[:]\n",
    "\n",
    "    \n",
    "    def back_propagate(self, x_train, label, lr, correct):  # cases = x_train[i]  label = y_train[i]\n",
    "        '''\n",
    "        反向传播算法\n",
    "        :param x_train: 训练数据\n",
    "        :param label: 标注\n",
    "        :param lr: 学习率\n",
    "        :param correct: 惯性动量系数\n",
    "        :return: 误差\n",
    "        '''\n",
    "        # 反向传播\n",
    "        self.predict(x_train)   # 得到的是输出层的输出\n",
    "        # 求输出误差\n",
    "        output_deltas = [0.0] * self.output_n  # [0, 0, 0]的数组\n",
    "        for o in range(self.output_n):\n",
    "            error = label[o] - self.output_cells[o]\n",
    "            output_deltas[o] = sigmoid_derivative(self.output_cells[o]) * error\n",
    "        # 求隐层误差\n",
    "        hidden_deltas = [0.0] * self.hidden_n\n",
    "        for h in range(self.hidden_n):\n",
    "            error = 0.0\n",
    "            for o in range(self.output_n):\n",
    "                error += output_deltas[o] * self.output_weights[h][o]\n",
    "            hidden_deltas[h] = sigmoid_derivative(self.hidden_cells[h]) * error\n",
    "        # 更新输出权重\n",
    "        for h in range(self.hidden_n):\n",
    "            for o in range(self.output_n):\n",
    "                change = output_deltas[o] * self.hidden_cells[h]\n",
    "                self.output_weights[h][o] += lr * change + correct * self.output_correction[h][o]\n",
    "                self.output_correction[h][o] = change\n",
    "        # 更新输入权重\n",
    "        for i in range(self.input_n):\n",
    "            for h in range(self.hidden_n):\n",
    "                change = hidden_deltas[h] * self.input_cells[i]\n",
    "                self.input_weights[i][h] += lr * change + correct * self.input_correction[i][h]\n",
    "                self.input_correction[i][h] = change\n",
    "        # 求全局误差\n",
    "        error = 0.0\n",
    "        for o in range(len(label)):\n",
    "            error += 0.5 * (label[o] - self.output_cells[o]) ** 2\n",
    "        return error\n",
    "\n",
    "    \n",
    "    def train(self, X_train, labels, epoch=10000, lr=0.05, correct=0.1):   # labels = y_train\n",
    "        # 训练神经网络\n",
    "        for j in range(epoch):\n",
    "            error = 0.0\n",
    "            for i in range(len(X_train)):\n",
    "                label = labels[i]\n",
    "                x_train = X_train[i]\n",
    "                error += self.back_propagate(x_train, label, lr, correct)\n",
    "        print(f\"error= {error}\")\n",
    "\n",
    "        \n",
    "    def fit(self, x_test):  # 离散预测函数用于输出数据\n",
    "        y_pre = []\n",
    "        for case in x_test:\n",
    "            y_pred = self.predict(case)   # return self.output_cells[:] ---> y_pred\n",
    "            for i in range(len(y_pred)):\n",
    "                if y_pred[i] == max(y_pred):\n",
    "                    y_pred[i] = 1\n",
    "                else:\n",
    "                    y_pred[i] = 0\n",
    "            y_pre.append(y_pred)\n",
    "        return inv_label_tr(np.array(y_pre))\n",
    "\n",
    "    \n",
    "    def fit2(self, x_test):  # 连续预测函数用于画图\n",
    "        y_pre = []\n",
    "        for case in x_test:\n",
    "            w = np.array([0, 1, 2])\n",
    "            y_pred = self.predict(case)\n",
    "            y_pre.append(np.array(y_pred).dot(w.T))\n",
    "        return np.array(y_pre)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':  # 主函数\n",
    "    data = np.loadtxt('Iris.data', delimiter=',', converters={4: trtype})  # 读入数据，第五列转换为类别012\n",
    "    X, y = np.split(data, (4,), axis=1)  # 切分data和label\n",
    "    pca = PCA(n_components=2)\n",
    "    X = pca.fit_transform(X)  # 为方便绘图，对x进行PCA降维至二维\n",
    "    y = label_tr(y)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1, train_size=0.8)  # 划分数据\n",
    "    \n",
    "    nn = BPNeuralNetwork()\n",
    "    nn.setup(2, 5, 3)  # 初始化网络\n",
    "    nn.train(X_train, y_train, 10000, 0.1, 0.1)  # 训练\n",
    "    \n",
    "    y_pre_1d = nn.fit(X_test)  # 测试\n",
    "    y_test_1d = inv_label_tr(y_test)\n",
    "    print(f\"测试分数为: {accuracy_score(y_pre_1d, y_test_1d)}\")  # 打印测试分数\n",
    "    \n",
    "    # 画图\n",
    "    x_min, x_max = X_train[:, 0].min(), X_train[:, 0].max()  # 第0列的范围\n",
    "    y_min, y_max = X_train[:, 1].min(), X_train[:, 1].max()  # 第1列的范围\n",
    "    xx, yy = np.meshgrid(np.linspace(x_min, x_max, 200), np.linspace(y_min, y_max, 200))  # 生成网格采样点\n",
    "    grid_test = np.stack((xx.flat, yy.flat), axis=1)  # 测试点  （xx.flat降维）\n",
    "    y_predict = nn.fit2(grid_test)\n",
    "    cm_pt = mpl.colors.ListedColormap(['r', 'g', 'b'])  # 样本点颜色（样本分为3个类，三个颜色）\n",
    "    cm_bg = mpl.colors.ListedColormap(['b', 'y', 'gray'])  # 背景颜色\n",
    "    mpl.rcParams['font.sans-serif'] = [u'SimHei']   # 设置字体为SimHei显示中文\n",
    "    mpl.rcParams['axes.unicode_minus'] = False  # 设置正常显示字符\n",
    "    plt.xlim(x_min, x_max)\n",
    "    plt.ylim(y_min, y_max)  # 设置坐标范围\n",
    "    plt.pcolormesh(xx, yy, y_predict.reshape(xx.shape), shading='auto', cmap=cm_bg)  # 绘制网格背景\n",
    "    plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_pt, marker='o')  # 绘制样本点\n",
    "    plt.title(u'BPNN分类', fontsize=15)\n",
    "    plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}